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ABSTRACT 

The usefulness of state-^by^state comparisons and 
other educational indicators is discussed in this paper, which seeks 
to improve report writing and data presentation skills. The first 
part offers an example of a policy report that uses key education 
indicators to describe the Arizona education system. Tables and data 
interpretation describe the system's size and diversity, fiscal 
resources, and student performance. The second part explains the 
policy analysis report process and how researcher values influence 
the interpretation and presentation of data. Problems in interpreting 
state rankings of student achievement are discussed. A conclusion is 
that state rankings help to reveal system strengths and weaknesses, 
shape dialogue and research questions, and ensure financial support 
for schools. Fourteen tables are included. (50 references) (LMI) 
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The rate of available information is accelerating. New magazines and books, regular television 
and cable channels, the ordinary and electronic mail, all putting increasing demands on the time to absorb 
information and energy to respond in a meaningful way. Policymakers are similarty inundated and have 
the responsibility of sifting through the calls and letters, newspapers, research reports and analysis, and 
then determining a course of action. "Keep it simple; one page or less" speaks to the difficulty 
policymakers have in controlling time and information ftow. Yet the request to put information succinctly 
does not explain how policymakers view and use information, be it from the public, press, researchers 
or policy analysts. 

The purpose of this essay is to assist educational researchers and policy analysts in making their 
work more available to policymakers by discussing how to view and how to think about educational 
indicators, parttoularly highly publicized findings such as state-by-state comparisons. The paper tries to 
make explicit the thinking that goes on when a policy report is prepared and draws on the authors 
experience in policy analysis at the Arizona DepartcDent of Education. The discussion of how 
researchers and policy analysts report education indicators such as state education rankings is intended 
to encourage a wore critical and reflective thinking by those engaged in the craft about their work, and in 
the end lead to t>etter connections between the research, analysis and the development of sound 
educational policy. As the nation begins a new round of state performance comparisons based on 
results of the National Assessment of Educational Progress (NAEP) the problems of making sense and 
drawing vaikJ meaning, for both analyst and policy maker, gain increasing importance. 

In order to accomplish this goal, the paper takes two directions. Rrst, an example of typical 
practice, one which summarizes key state education indicators, is presented. Tables and discussion are 
adapted from a report by Danzig and Mungazi done for the Arizona Townhall Academy, a group of 
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business, community and educational leaders who meet on an annual basis to on a selected topic/ The 
topic selected for 1989 was education and the title of the Report was s o s Save Qur Schools. Save 
Our state . The work is similar in underlying approach to policy reports done at many state department 's 
of education and Includes discussion of the education system in Arizona . education indicators, and 
selected comparisons with other states. The second part of the paper looks more to the concept of 
usefulness and discusses how policy analysts prepare their reports and some of the values that are part 
of a seemingly straightfoiward reporting of "facts." Values, in this sense, refer to reasons why certain 
problems are approached, questions asked and comparisons accomplished. Ultin^ately, it is hoped this 
will shed some light for others in the reading and interpreting of education reports and state-by-state 
comparisons. 

State-Bv-State Eduftation Comparisnnfi 

A number of federal agencies provide data from which state-by-state comparisons are made. 
The U. S. Department of Education's Center for Education Statistics annually publishes The Digest of 
Eduratinn Statistics and The Condition of Education , which contain educational statistics Of interest to 
educational leaders and policymakers that are reported on a national and state-by-state basis. Beginning 
in 1984, and every year since, the Office of Planning, Budget and Evaluation in the U.S. Department of 
Education has published State Education Statistics or more comnx>niy referred to as the "Secretary's Wail 
Chart," a summary of key indicators concerning population characteristics, resources, and educational 
achievement for each of the states and presented in a large poster or chart format. The U.S. Commerce 
Department , Bureau of of the Census also reports govemment and school finance data from which state 
ranking is possible.^ 

National and regional organizations also publish education statistics which include state 
rankings. The National Education Association compares and ranks states on variables deemed of interest 
to teachers Including salary and per pupil expenditure data ^ The Western Interstate Commission for 
Higher Education (WiCHE) focuses on higher education but includes analysis of demographics and 
teacher supply and demand data for states in the westem region of the United States.^ The Southern 
Region Education Board compares states in the south on education indicators with particular focus on 



measuring pupil achievement.^ Beginning in 1988, the Council of Chief State School Officers published 
state Edueatinn Indicators . a summary of demographic and fiscal backround information bearing on the 
states' investiment in schooHng.^ Plans to look at comparative indicators of student outcomesbased on 
an expanded National Assessment of Education Progress are currently in the works. 

Finally, public and private groups provide comparative analysis of state education indicators. The 
Institute for Educational Leadership k>oks at demographic indicators and compares states on selected 
education variables such as retention, teacher salary and per pupil expenditure/ On request, lEL will 
provide a sumniary of key indicators targeted at a specific state. Policy Analysis for California Education, 
funded by the Hewlett Foundation and jointly directed by professors at the University of California 
Berkeley and Stanford University , provides analysis of education indicators and trends for California 
schools and often includes comparisons with other states.^ 

The preceding shouki give some indication of the large numbers of organizations and individuals 
interested in knowing not only In how education fares in a particular state, but also in how a state compares 
with other states, be it states of similar size, economic diversity, geographic region or some other basis of 
comparison. And it woukl seem logical that each group that aocompishes an analysis has its own 
purposes and audience. The next section presents an example of typical practice that spotlights Arizona 
in its discussion of education. 

TYPICAL PRACTICE: DESCRIPTION OF SIZE AND DIVERSITY 
In describing a state's education system, infomnation is presented concerning the number, size 
and diversity of schools, districts, students and teachers. Such information tells the reader who is 
Involved and how many there are of them. Selected variables such as size (of schools, of districts) and 
geographic location (urban, mral) permit contrasts and comparisons among localities to be made. Since 
data are collected on a regular basis (for the most pari annually), the writer is able to present a view of 
selected variables over time and provide the reader the opportunity to k>ok at possible implications of 
such changes. 
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Number and Size of Ari?Qna Schools and Districts 
During fiscal year 1986-87, there were 217 school districts in Arizona conslstir)g of 781 
elementary schooK and 151 high schools.^ Of this number, there were 14 one-room schools. Table 1 
provides data on district size and the reader should note the relatively large number of small districts (45 
districts with enrollment less than 100). 



Table 1 
Arizona School District Size 
[In Average Dally Membership fADM^I 



ADM 


#of Districts 


Percentage 


Less than 100 


45 


20 


100-600 


55 


25 


601 -1,000 


28 


13 


1,001 -5,000 


64 


29 


6,001 • 10,000 


11 


5 


10,001 - 20.000 


10 


5 


20,001 - 30,000 


2 


1 


Over 30,000 


2 


1 



Source: Arizona Department of Education. School Finance Fact Sheet. 1987e. 



The reader should also note an Increase In the number of schools. In fiscal year 1987-1988, the total 
number of elementary schools had increased to 798 (up 17 schools) and the number of high schools had 
Increased to 159 (up 8 schools) . At the same tln^ the total number of districts declined by seven, from 
217to 210 districts."" 0 

Schools on the Reservation 

Ten school districts (Chinle, Ganado, McNar>;, Red Mesa, Wlrxiow Rock, Tuba City, Cedar, 
Kayenta, Whiteriver, and Sacaton) with 34 schools serving approximately 1,900 students, are located 
totally on a resen/ation. An additional 29 school districts have one or more schools located on a 
resen/ation."' ^ 

4 r 
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Demographics 

Arizona continues to experience population growth. Between 1970 and 1985, Arizona's 
population shows 79.5 percent growth, compared to a national growth rate of 17.4 percent. Between 
1985 and the year 2000, Arizona's population growth is projected at 70 percent, compared to a 12.2 
percent population growth forecast for the nation as a whole. 

Arizona's population is also somewhat younger than the national average, indicating higher 
educational needs. In Arizona, 8.5 percent of the population is under five, compared to 7.6 percent for 
the nation as a whole. According to the U.S. Department of Education, the school-age population for 
Arizona is 604,000 or approximately 19.0 percent of total population in Arizona.''^ This compares with 
18.8 percent for the nation. One implication is that there will continue to be strong growth in the school- 
age population during the coming decade. 
Growing Minority Poptjlations 

Minority populations are growing as a proportion of the total population ot Arizona. Table 2 
indicates the extent to which there is changing demography in Arizona, with particular emphasis noted for 
the population of age five and younger. 

Table 2 

Changing Demography In Arizona 





1980 


1980 


1986 


1986 


Ethnicitv 


All Ages 


Under Age 5 


AZK-12PuDils 


Births 


Black 


2.0% 


3.3% 


4 0% 


3.4% 


Spanish Origin 


10.7% 


25.6% 


22.3% 


34.7% 


American Indian 


3.5% 


9.2% 


6.5% 


6.2% 


White (Non-Hisp.) 


83.1% 


61.0% 


65.8% 


54.6% 


Others 


0.7% 


0.9% 


1.4% 


1.1% 



Sources: 1986; Arizona Department of Education. School FinanoA Factsheet. iaaa. 



Among all ages of the Arizona population in 1980, 83.1 percent were Anglo, 10.7 percent IHispanic origin, 
and 6.2 percent other minorities. Among age five and younger in 1980, 61.0 percent were Anglo, 25.6 
percent Spanish origin, 9.2 percent American Indian and 3.3 percent Black. Among new births in Arizona 
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in 1986, nearly one-haK were from minority populations with 34.7 percent of Arizona Idirths in 1986 being 

Hispanic. This represents a changing school age population. 
Primary Languaqa and Arizona Students 

One Of the issues raised by the changing ethnicity of the population is the increasing language 
diversity of Arizona pupils. Table 3 indicates the primary home language of students enrolled in Arizona's 
public schools during the 1985-66 school year and shows that slightly more than 1 6 percent of Arizona 
pupils cite a language other than English as their prlriiary hdme language. For certain counties, i.e., Santa 
Cruz (82%), Apache (55%), and Yuma (40%), the figures are much higher and pose special challenges for 
the public schools to meet. 



Table 3 

Primary Home Lan?ua^ie gnroliment bv noiintv 
Arizona Public Schools - 1986 



County 


English 




Indian 


Othtv 


Total Non-English 


%Non.E 


Apache 


6,241 


49 


7,372 


21 


7,432 


65% 


Cochise 


4,930 


4,804 


7 


339 


5,150 


26% 


Coconino 


13,619 


259 


3,941 


82 


4,282 


24% 


QIa 


6,634 


440 


718 


10 


1,168 


15% 


Graham 


4,619 


287 


263 


1 


551 


11% 


Greenlee 


2,208 


99 


6 


0 


105 


6% 


1^ Paz 


2,376 


599 


4 


8 


611 


20% 


Maricopa 


98,044 


29,628 


880 


3,792 


34,300 


26% 


Mohave 


13,553 


137 


257 


69 


463 


3% 


Navajo 


1 1 ,698 


379 


4,811 


29 


5,210 


31% 


Pima 


80,779 


18,061 


250 


1,816 


. 20,127 


20% 


Pinal 


19,497 


3,080 


584 


39 


3,703 


16% 


Santa Cmz 


1,134 


5,023 


0 


36 


5,069 


82% 


Yavapai 


13,971 


444 


90 


30 


564 


4% 


Yuma 


1 1 ,865 


7,656 


101 


73 


7,830 


40% 


TOTALS: 


501,165 


71,045 


19,284 


6,345 


96,674 


16% 



Source: Arizona Department of Education. Bilingual Programs and English as a Second Language 
Program s. 1987d. 



In summary, the data suggest a relatively large number of small school districts, a large number of 
resen/ation schools, and a large number of one-room schools. Oenr>ographic trends indicate a higher than 
national average birth rate, and a growing number of minority school-age children. Finally, a high 
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proportion of Arizona's school-age population is seen as coming from homes with a primary language 
other than English. One might corxjiude that as a state. Arizona has a rapidly expanding and changing 
population the implications of which are yet to be explored. 

TVPICAL PRACTICE: DESCRIPTION OF FISCAL RESOURCES 

This next section attempts to answer five questions concerning the financing of elementary and 

secondary education In Arizona. Information is presented to answer questions concerning source of 

funds, revenues, per capita income, and per pupil expenditures. Arizona is compared to its neighboring 

states in the Western region of the United States (California. Colorado. Nevada. New Mexico, Texas and 

Utah) and the national average. In addition and based on the request of the Townhall committee, data on 

the state ranlted first (#1) in the nation are included as appropriate. The logic of comparisons states is 

based solely on the concept of neighboring states. 

Snurrpg of Revenue 

Table 4 indicates the three major sources for financing the public schools: federal grants, state 
appropriations, and local property taxes. Over the past fifteen years there has been a major shift in the 
source of funds to support public education, from the local to state levels. In 1970, for example, 48 
percent of the funds to support public education in Arizona came from local sources, compared to 45 
percent from state appropriations. In 1985. 32 percent came from local tax revenues while almost 59 
percent camo from state appropriations.''* This shift is consistent with trends in the region and nationally. 
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Tabl6 4 

Distribution of Revenue Receipts of Publle Elemerrtary/Seeendary Schools 
tor Selected States: Federal. State & Local Sources (in percent) 



Federal State Local 



state 






1970 




im 


1965 


Arizona 


7.6 


6.9 


44.6 


58.7 


47.8 


32.4 


California 


5.1 


7.9 


35.2 


67.5 


59.8 


24.5 


Colorado 


7.3 


4.7 


30.3 


39.6 


62.4 


55.8 


Nevada 


16.6 


5.4 


63.4 


40.3 


20.0 


54.3 


New Mexico 


6.9 


12.7 


36.8 


75.0 


56.3 


12.3 


Texas 


10.0 


6.6 


49.3 


47.1 


40.7 


46.3 


Uah 


5.5 


5.9 


54.7 


51.8 


39.8 


42.3 


U. S. Average 


6.9 


6.5 


41.1 


48.8 


52.0 


44.7 



Source: U.S.Department of Education. Digest of Edu cational Statig*ics. laa?. 



Expenditures and Revenues for Education 
Three different groups currently report data concerning expenditures for education: the U.S. 
Department of Education, the U.S. Bureau of the Census, and the National Education Association. 
Depending on the source of the data used, different interpretations of where Arizona ranks on per pupil 
expenditures (PPE) are possible. Differences in ranltings can be attributed to errors in the reporting of 
state data, decisions of what to include in calculating educational expenditures, and ways of measuring 
pupil attendance.is 

Table 5 indicates the expenditure per pupil according to the U. S. Department of Education. The 
data show that while per pupil expenditures for education have increased in Arizona, it has not kept pace 
with other states in the region or with the national average. 
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Table 5 

Current gxpendlture Per Pupil In Averafffl Dally Attendance In 
Public Elementary and Setaondary Schools, by Souttiwestern State: 
School Years ending 1970 and 1986 

Expenditure Per Pupil 
(1985-86 dollars) 



Slals. 1^ 1986 Percent Increase 

Arizona 2.075 3.093 49.1% 

Califomia 2,496 3.543 41.8% 

Colorado 2.126 3.975 • 87.0% 

Nevada 2.217 3.440 55.2% 

New Mexico 2.037 3.195 56.9% 

Texas 1.798 3.298 83.4% 

Utah 1.804 2.390 32.5% 

#1 Alaska 3.234 8.253 55.2% 

United States Average 2.351 59.6% 



Source: US Departnnent of Education. The Condition of Edupation. I98fl. 



One possible interpretation of the data is that Arizona is having difficulty keeping pace with the 
many pressures of rapid population expansion and that this is reflected in slower growth rates in per pupil 
expenditures than slower growth states. Also, school expenditure data often exclude bonded 
indebtedness (used to pay for new school buildings and expansion of existing facilities). Once voters are 
asked in bond elections for nnonies for school buildings and expansion they may be less willing to vo'.e for 
additional taxes to pay for higher teacher salaries or more expensive instructional programs. This is one of 
the hidden costs of being a state with significant population growth. 

Revenues in Relation to Per Capita Income 

Revenues in relation to per capita income speaks to the issue of how much money is available to 
elementary and secondary public schools. Table 6 examines per pupil education revenues, a different 
figure than per pupil expenditures, and relates it to population, personal income and per capita personal 
Income. Arizona is last among the compared southwestern states and ranks 49th in the nation In this 
category, with Alabama the only state with a k)wer state index. While the wealth of the state as measured 
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by per capita personal income ranks in the nr>iddle of the 50 states, public school revenues are nearer the 
bottom. T.-^ken together, this ratio ranks Arizona 4C!h in the nation. 



Table 6 

State Inriiftftfi of Public School revpnuRs Per Pupil Relation 

to Per Capita Income: School Vaars Ending 1980 and 1987 

State & kx:al Public Total 

education alementary/ Per pupil personal Total Per Capita 

revenues secondary education Income populbtion personal 

State Index (thousands) enrollment revenues (milKons) (thousands) income 

1980 1987 1987 Fall 1986 1987 1986 1986 1986 



Arizona 


25.1 


19.3 


1,387,060 


534,536 


2,595 


44,719 


3,31 S 


13,474 


California 


21.6 


22.2 


16,463,619 


4,377.989 


3,761 


456,098 


26,981 


17,472 


Colorado 


26.9 


28.6 


2,433.400 


558,415 


4,352 


49.771 


3,267 


15,441 


Nevada 


18.2 


21.7 


540,519 


161.239 


3,352 


14,870 


963 


11.423 


New Mexico 


25.2 


27.3 


880,451 


281,943 


3,123 


16.894 


1,479 


11.423 


Texas 


20.4 


26.9 


11,648,726 


3,209,515 


3,629 


224.8/7 


16,685 


13.478 


Utah 


24.2 


23.6 


1,088,471 


415,994 


2,617 


18,288 


1,665 


10.984 


#1 Wyoming 


25.7 


54.5 


703,950 


100,955 


6,973 


6.485 


507 


12.791 



Source: U.S. Department of Education. The Condition of Education . 1988. 

Attendance. Enrollment, and PuPiUTeacher Ratic 
Two additional educational indicators are compulsory attendance requirements and pupil-teacher 
ratio. Table 7 tooks at ages for compulsory attendance laws. Arizona, provides for the fewest years of 
compulsory schooling of any state in the region and, although not shown, the fewest years of compulsory 
schooling of any state in the nation. A decision to require an additional year or two of compulsory 
attendance (from age 16 to age 17 or 18) wouki have obvious and immediate fiscal implications. 
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Table? 

Ages for Com pukftfv fifthool AttAndancfl. fay Stata: 1985 



Compulsory Attandance 

Arizona 6 to 16 

CalKomla 6 to 16 

Colorado 7 to 16 

Nevada 7 to 17 

New Mexico 6 to 16 

Texas* 7 to 16 

Utah iBtolS 



* Must complete academic year In which 16th birthday occurs. 
Source: U.S. Department of Education. Dinest of Educ atior^ Statistics. 19S7. 



Table 8 addresses the issue of pupil-teacher ratio. Although it is not possible to directly connect 
pupil-teacher ratio with educational performance, it Is an additional indicator of a state's expenditure on 
education. Comparing ".982 and 1987, Arizona has improved, on the average, with smaller class size and 
a reduced pupil-teacher ratio (from 1 9.8 to 18.4). Compared with other states, Mzm ranks 36th 
nationally, up from 37th in 1982. 



Table 8 

Pupil-Teacher Ratio. 1982 and 19B7 



State 




(RanbO 


mi 


(RanK) 


Arizona 


19.8 


(37) 


18.4 


(36) 


Califomia 


23.1 


(50) 


23.0 


(50) 


Colorado 


18.7 


(30) 


18.2 


(33) 


Nevada 


21.1 


(46) 


20.4 


(46) 


New Mexico 


18.8 


(31) 


19.0 


(41) 


Texas 


18.4 


(25) 


17.3 


(26) 


Utah 


27.4 


(51) 


23.4 


(51) 


#1 Connecticut 


15.0 


(1) 


14.0 


(1) 


US Average 


18.9 




17.8 





Source: U.S. Department of Education. State Education Statlstlcs.l98B. 
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To Sum, when Arizona is compared with other states and the nation on indicators such as 
revenues per pupil, expenditures per pupii. and years of compulsory attendance, Arizona ranlts below the 
national average. With regard to the ratio of a states per capita income compared to public scriool 
revenues per pupil, an indicator interpreted to reveal the comnoittment of a state to its schools, Arizona 
ranks 49th: concenring required years of schooling, Arizona ranks last in the nation. One noted 
exception for Arizona schools has been the pupil-teacher ratio, where Arizona compares somewhat more 
favorably (36th). 

TYPICAL PRACTICE: DESCRIPTION OF STUDENT PERFORMANCE 
This section looks at student performance on a nurnber of educational outcomes. Descriptive 
and comparison data are presented for five measures: (1) college entrance examinations; (2) Merit 
Scholarship Awards: (3) Advanced Placement tests; (4) standardized achievement tests; and (5) high 
school graduation. As with the previous section, comparisons are made with neighboring states with the 
state ranked first and the national average given where approapriate. 

Collftgfi Entrance Examination Scores 

One indicator of student achievement is scores on college entrance examinations, namely, the 
Scholastic Aptitude Test (SAT) and the American College Testing Program (ACT). The national trend in 
college entrance examinations show a decline in scores beginning with the 1963-1964 school year that 
continued through the 1979-80 school year. Though much of the decline in early years has been 
attributed to increases in the percentages of lower ability students taking the examinations, decline in 
scores in the later years is more attributed to a collection of factors, including school relevance and rigor.'* ^ 

* 

Table 9 k>oks at the number of Arizona students taking one of the college admissions exams (ACT 
or SAT). Table 10 tooks at ACT scores only, because the majority of students in Arizona (and the five 
states listed) take the ACT rather than the SAT. 
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Table 9 

Amerinan ftolleoe Testing Program (ACT) and Scholastic Aptitude Test 
(SAT) fof Arizona High Setionl Students 



Year Number of Students Number of Students Total 

Taking ACT Taking SAT 

1984- 85 11,547 4,402 15,949 

1985- 86 10,934 5,045 15,979 

1986- 87 11,214 6,463 17,677 

1987- 88 11,549 7,039 18,588 



Source: Arizona Department of Education. Statewide Report for Arizona Pupil Achievement 
Ifi&tins^1988. 



Table 9 indicates that the actual number of Arizona students taking either or both examinations for 
college admission has gone up since 1984-85. However. Table 10 indicates that the percentage of 
high school graduates taking the ACT has actually declined since 1982. This decline is not 
accounted for by students opting for the SAT examination. 

Table 10 

American Collepe Test Scores: 1982 and 1QR6 

Average ACT Score Percentage of High Average ACT Score Percentage of High 
of High School Grads School Grads Taking of High School Grads School Grads Taking 



1986 Testinl986 1982 Testinl982 



Arizona 19.3 38.2 18.7 41.2 

Cotorado 19.9 63.4 19.6 66.8 

Nevada 19.1 40.5 18.3 44.5 

New Mexico 18.0 53.9 17.6 56.5 

Utah 18.9 67.2 18.4 66.4 

#1 North Dakota 18.8 68.3 17.8 64.5 



U.S. Average 18.7 18.4 
Source: US Departnient of Education. State Education Statistics .iQaa. 

Table 10 also shows that the average ACT score of high school graduates in Arizona has gone up 
since 1982 from 18.7 to 19.3. However, it is unclear whether the increase is attritHitable to the decline in 
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the percentage of students taking the exam. And. the percentage of high school graduates taking the 
Test In 1986 (38.2%) ranks Arizona 26th out of the 28 ACT states . 

Achievement of Gifted and Talented Students 

Another Indicator of educational outcome Is how the brightest or most able students are 
achieving. Two sources of data are relevant to this discussion: (1) student performance In the Merit 
Scholarship Program, and (2) student participation in advanced placement courses. 
Merit Scholars 

According to Eckland the National Merit Scholarship Corporation was founded in the middle 
1950s to Identify high school students "who wouki merit special commendation and economic 
incentive".''^ In 1971, the National Merit test was combined with the Preliminary Scholastic Aptitude Test 
(PSAT) and eleventh graders wishing to qualify must take what is called the PSAT/NMSQT. For college 
admissions, students would still be expected to take either the SAT or ACT (or both) as twelfth graders. 
Table 1 1 tooks at the qualifying selection scores and the number of commended and semifinalist students 
recognized by the Merit Scholarship Program. 

Table 11 

1988 Merit Program 
Semlfinallsts and Commended Students - 19B6 PSAT/NMSQT 

SEMIFINALISTS COMMENDED STUDENTS 



Qualifying Selection 


Number 


Number of 


Number 


Number of 




Index 




Schools 




Schools 


Arizona 


195 


163 


54 


308 


81 


Callfomia 


198 


1,421 


440 


3,442 


718 


Cok)rado 


196 


196 


68 


498 


131 


Nevada 


188 


47 


21 


64 


25 


New Mexico 


195 


99 


27 


169 


41 


Texas 


196 


912 


227 


. 1,963 


453 


Utah 


195 


116 


34 


200 


50 



Source: National Merit Scholarship Corporation. Annual Report. 1987. 



Table 1 1 gives Information conceming the qualifying scores for students in Arizona and other 
states In the region, the numbers of students identified as a "semifinalist" and those identified as 
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"commended." Commended students are those achieving excellence, although no qualifying score is 
given and no financial incentives are provided. The National Mem Scholarship Program does not reveal 
the percentage of semi-finalists who are actually awarded scholarships, in part this dedsion is based on 
examination score, the college atte.ided, and a student's background variables. A comparison of 
qualifying scores shows Arizona to ' a tied for fourth in the rogion, where only Nevada's students qualify 
with lower scores. Similarly, the number of students identified appears lower than expected if one 
compares it to Colorado, which has almost the same school age population (see Table 6). 
Advanced Placament Participation 

High school students across the nation are given the opportunity to earn college credit through 
Advanced Placement courses. Table 12 presents data on advanced placement candidates as a percent 
of high school graduates. 

Table 12 
Advanced Placement Candidates 

(as a percent of graduates) 





mi 


(Rank) 


mi 


(Ranl<) 


Arizona 




(28) 


3.5 


(25) 


'Callfomia 


.0.7 


(6) 


8.1 


(5) 


' Colorado 


13.1 


(10) 


8.1 


(5) 


Nevada 


10.2 


(16) 


1.7 


(32) 


New Mexico 


8.8 


(22) 


1.7 


(32) 


Texas 


5.2 


(35) 


1.9 


(30) 


#1 Uah 


26.6 


(1) 


11.8 


(1) 


US Average 


9.7 




4.7 





Source: US Department of Education. State Education Statistics .1988. 

The data indicate that while Arizona has more than doubled its percentage of qualifying students, 
this figure has not kept pace with regional or national trends. And in fact. Arizona has fallen from 25th 
nationally to 28th nationally over the five year period 1982-87.1^ 
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Pupil Aehlevement Testing 

Another way of looking at educational outcomes is to look at the results of the tests administered 
in the 1988 Arizoiu: F^pil Achievement Testing Program, namely the ITBS (grades 1-8), the SAT-7 (grade 
9). and the TASK (grades 10-12). These are natior^lly standardized achievement tests that have been 
administered under specific conditions to a representative sample of students across the nation. Scores 
that result from the rwrming sample allow comparisons of the per1omf>ance of particular pupils or groups of 
P4:pils *o that of the typical pur)ll of the same age and grade. 

Arizona is the only state in the nation which requires that all students from 1st through 12th 
grades take a standardized achievement test every year. Legislation passed in 1988 (SB 21 11) permits 
local districts, for the first time, to decide whether first or twelfth graders are to be tested. Approximately 
one week oui of the school year and ctose to $1 million per year represent some of the resources devoted 
to this testing program. 

Results uf the 19Rfl Arizona Pupil Achievflment Testing Pronram 

J\M,i Arizona Department of of Education reports that Arizona students achieving in the average 
range: ^Arizona pupils collectively scored in the average range in all areas tested on ail tests taken (ITBS, 
SAT-7. and TASK) in all twelve grades.i^ This is evidence of the fact that Arizona pupils perfomned at 
about the same level of achievement as average pupils in grades 1 through 1 2 across the nation."^^ The 
tests are composed of numerous subtests combined to form subject or content domains such as reading, 
language, und mathematics. Although the tests claim to measure a broader base than just these subjects, 
the Arizona legislature has mandated testing of reading, language, and mathematics. 

Another way to interpret the overall perfom^nce of Arizona pupils is to look at average stanines. 
Of the 124 skills and areas measured across grades 1 through 8, Arizona pupils, on a scale of 1 to 9 (with 
an average scoic being 5), obtained an average stanine of 5 on 98 skills (79%) and an average stanine of 
6 In the remaining 26 skills (21%).^^ 



High SehoQl Graduation 

Another indicator of performance is graduation rate from high school. According to Hodgl<lnson, 
high school graduation is directly related to state-level economic development.^^ A state with a high 
percentage of its youth graduating yields a "net gain" with a high prol)ability of that person getting a job 
and repaying the state many times over for the oost of his/ho» education, through taxable income. On the 
other hand, a state with a poor record of retention to high school graduation, suffers a "net loss" for the 
state, because without a high school diploma, the chances of a student's finding work and thereby 
repaying the state's investment is diminished. 

Table 1 3 provides data for Arizona's high school graduation rate and provides comparison data 
from other states in the region. 

Table 13 

High School Graduation Rate (Adjusted) 



State 


1982 


Rani< 


1985 


Ran>< 


1986 


Rank 


Arizona 


63.4 


(44) 


64.2 


(42) 


63.0 


(47) 


Califomia 


60.1 


(49) 


67.0 


(39) 


66.7 


(41) 


Colorado 


70.9 


(29) 


74.0 


(28) 


73.1 


(28) 


Nevada 


64.8 


(40) 


70.9 


(34) 


65.2 


(42) 


New Mexico 


69.4 


(33) 


73.4 


(30) 


72.3 


(29) 


Texas 


63.6 


(43) 


66.0 


(41) 


64.3 


(44) 


Utah 


75.0 


(17) 


81.4 


(10) 


80.3 


(12) 


#1 Minnesota 


88.2 


(1) 


91.5 


(1) 


91.4 


(1) 


U.S. Average 


69.5 




71.7 




71.5 





Source: US Department of Education, state Education Statistics. 1988. 



Arizona's graduation rate falls below the national average and rani<s near the bottom of the nation 
(47th out of 51 in 1986); its relative standing anwng the states has declined during the past five years. 

Non-Promotion 

A related but separate issue deals with promotion from grade to grade. Shepard and Smith report 
that retention in grade has no benefits for either school achievement or personal adjustment.23 Yet, as 



17 



18 



Doyle reports.during the 1986-87 school year 15,183 Arizona elementary school children failed to be 
promoted to the next grade.^^ At $3,093 (1986 per pupil expenditure figure}per child the costs 
associated with providing an extra year of schooling is approximately $45 million, in spite of the the 
research evidence that shows non-promotion as having hannful effects to students (i.e., students 
malting slower academic progress than comparable under-achieving students; subsequent dropping out 
of school) the percentage of non-promoted elementary students In Arizona Is increasing. 



Table 14 

Percent of Non-Promotion in Arizona 
Elementary Schools since 195Q 
(averaged for five year periods) 



1950-1954 4.3% 

1955-1959 4.1% 

1960-1964 3.4% 

1965-1969 3.3% 

1970-1974 2.4% 

1975-1979 3.2% 

1980-1984 3.5% 

1985-1987 3.8% 



Source: Doyle, 1988. 

Looking at the data over eight five-year blocks of time. Table 14 shows the percent of non- 
promoted elementary pupils as increasing since 1970-74. For those who believe that the cure (non- 
promotion) may be worse than the disease (underachievement in school), the rise in the percentage of 
rK)n-promoted students since 1974 is suggestive of an education system failure in Arizona. 

To sum, the educational performance of Arizona pupils on a number of indicators is described. 
Higher ACT scores are mitigated by the fact that a snruiller percentage of students took the exmaination in 
1986 than in 1982 . Results of the National Merit Scholarship Exam indicate lower cutoff scores and 

* 

fewer semifinalists and commended students when Arizona is compared with Colorado, a state with a 
similar number of students. Graduation rate ranks Arizona 47th in the nation and relative standing has 
declined over the past few years. Advanced placement participation, while on the rise, places Arizona 
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below the national average and Arizona ranks lower among the stat^>s» in 1987 than 1982. Pupil test 
scores reveal, at best, 'average ' performance. Finally, a rise in pupil non-promotion is ciocumented. 

MAKING SENSE OF STATE COMPARISONS: TME ANALYSTS' VIEW 

At first glance, the data contained In the description of typical practice do not present a 
particularly optimistic picture of Arizona's educational system. The conclusion of an analysis of the data 
presented in Tables 1-14 by Danzig and Mungazi suggested that lack of financial resources necessary to 
keep up with population growth was at the root of academic decline in Arizona and that recognition of this 
fact was a required first step in doing something about it: 

"Although dollars will not guarantee educational success, they may be a required first step 
to ensure the success and assure the k)ng term future of the schools within a state. 
Arizona, a state with a growing pcpulation and higher than average growth In the school- 
age population, and with Increasing numbers of ethnic and language minority students, 
faces particular difficulties. Per-pupil revenues and expenditures, pupil-teacher ratio, and 
compulsory years of schooling all place Arizona no higher than the bottom third and 
sometimes last in the nation. The consequences seem to be average pupil 
achievement, toss of some of the most gifted and talented students to out-of-state 
universities, low and decreasing high school graduation rates, and Increasing 
percentages of non-promotion In the elementary grades. Recognition of and agreement 
on these realities may be the first step in reversing the downward trends ."^^ 

• 

However, this is only one possible interpretation of the data and does not rule out alternative 
explanations. More importantly, it does not address the way researchers select information and data which 
confimis their own expectations and those of policymakers. Since thf< reader receives no assistance in 
tiow to read or reflect upon the data, the elegance, clarity and pen- )S volume of the statistics contribute 
to the readers sense of accuracy and the veracity of its interpretation. 

What is Useful about State r^nm parisons and Education InHirainrs? 
While rankings and other education indicators are Intended to inform policymaking, it is not clear 
how policymakers use them to define courses of action. One way to judge usefulness may be to look at 
how state comparisons stimulate debate over problems, issues and solutions. In this view, state-by- 




state rankings reduce complextty and focus debate; simpdclty arxl efficiency are their virtue as they make 
issues appear more manageable. State comparisons also functton as symbolic and poUtical gestures In 
order to make the schools appear to be accountable, without which, public willingness to spend more 
nwney on education may be reduced. State con^parisons Inforni us of threshoWs, of how far the system 
needs to come before H reaches the top and of why the goal has not been attained. Finally, where 
rankings provide multiple evidence points and altow fortriangulatlon. the likelihood of accurate 
Interpretation may be Increased. Each of these views are exptored In greater detail below. 

Education Ranklnns As f^nmrrvintaatlon anri nkry..|r«|ft 

The effort to collect educational statistics as a basis of Informing education policy Is not new. 
Travers points to the wortt of Horace Mann In Massachusetts and Henry Barnard In Connecticut as laying 
the foundation for federal efforts to systematically collected education-related data.26 Travers suggests 
the motive and guiding principle behind the establishment of a central collection point for data was the 
view that "policies should be guided by empirical infomiation.- 27 Mumane. also tooks at federal efforts to 
colled educational data from which state-by-state comparisons couW be made.28 While enrollment 
statistics were collected In the I9th century, it was during the 1950s and 1960s, that the federal role was 
extended to Include Information about what students learned In schools. In this view, educational 
rankings provided an authoritative and socially relevant knowledge which In turn affected decision making. 

Cohen and Garet suggest viewing research as a form of discourse.29 The notion of research as 
discourse calls attention to the norms of communication which affect decision making, and to new 
problems and Issues such discussion generates. Research, data collection and reporting are not simply 
technical activities or a better way of deciding what to do. Rather, the Importance of such wort< is in 
Influencing broad assumptions and opening up new questions as well as answering okJ ones. The goal of 
such discourse is to open up communication and ways of focusing on key issues. 

State-by-state rankings provkle a fomm from which It Is possible to debate the meaning and 
Importance of not only the findings but of the questions asked (and those not asked). Federal reporting 
of state education statistics such as 8tate-by-s*ate rankings are not simply amassing of tmths 
Indiscrimlnantly; they are selective and simplicity of presentation is its virtue. The greater the complexity 
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of facts, the less chance of surviving. Volume of information is also related to usefulness. The Annual 
Rftpnrt of the Supflrintandflnt of Public instmction provides an example.^o In Arizona, such reporting 
goes titack to the 1880s, is required by statute and provides education statistics on school enrollment, 
expenditures and budget. Since data is collected on a regular basis (for the most part annually), in 
addition to answering questions about who and how many, the Report provides the reader the 
opportunity to make comparisons not only between groups, but to \ook at changes over time and 
possible implications. Though seenf)ingly straightforward, the volume of infonnation (241 pages), format 
of presentation ( mostly school district, county and state summaries of budget printouts) affects who will 
read a report and how it is distributed. In 1987. only 625 copies of the document were printed.^^ In 
contrast, one finds a more limited scope (24 pages) and fewer tables (9) in the report by the Arizona 
Department of Education comparing Arizona and the top ten states on per pupil exper\diture:32 yet 3000 
copies were printed and disributed, at no cost. 

If the usefulness of state-by-state comparisons is seen in their influence over state level policy 
debate, and complexity and volume r^* !:iformation affect discourse, distribution of ideas and potential 
impact, smaller and simpler are better. The debate over per pupil expcv-tditures (presented in Tables 5 
and 6) has in fact generated tvt/o statistical reports by the Arizona Department of Education which compare 
Arizona to other states and explain why Arizona ranks so low in the federal reports. And the Govemor 
recently formed a statewide task force "Fiscal 2000" to address funding issues for education which 
include per pupil expenditures. Again, it is the the reduced scope of reporting on per pupil expenditures 
(rather than school finance in general) which makes it more useful in furthering public discourse on the 
topic. 

Rankings and Financial Support for the Public Schools 

Kirst points out that public perceptions and willingness to finance public education are often 
dependent on issues outside the public schools.^ In the 1950s Sputnik, in the 1960s and 1970s, the 
War on Poverty, and in the 1980s, concern with American competitiveness abroad, have fueled 
educational reform. Between 1970 and 1980. state governments increased their total spending on 
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education from $16.6 billion to $46.5 billion, an increase of 44.5 percent in real dollars. State share of 
funding rose from 37 percent to almost 50 percent, while local and federal shares declined. Also, per 
pupil expenditures rose by about 9 percent in real terms during 1983, and increased faster than inflation 
during 1984 and 1985.^^ Kirst suggests that states will not be able to simply spend more money on 
education; if educational refomfi is to be successful, then effort is needed to determine which reforms of 
the past ten or fifteen years have been most successful and which reforms seem to be wore cost-efficient. 
In Kirst's view, educational rankings that reflect pupil performance (Table 10 on ACT scores, Table 12 on 
Advanced Placement candidates and Table 13 on high school graduation rate) provide some of the data 
needed by state policymakers to argue for increased funding for education. 
Avoiding Thresholds 

Education indicators may also function as warning signals. Vtckers points out that when we drive a 
car, we are controlled not so nrtuch by an imaginary course but by tho warning signals as we nx>ve to the 
threshokl of our driving track.^^ We set out therefore, not so much to to hold a course but to avoid the 
thresholds. The difference between pursuing a track and avoiding a threshr>ki are many. For one, "if we 
pursi." u nomn we may eventually attain it and thus discover whether we really like It: whilst, so tong as we 
avoid a threshokl, we can never tell, except as the result of failure, whether it deserved the respect we 
accorded it."^^ State education rankings function as the threshokl. Being ranked first, regardless of what 
it means to be first or how a state achieves this goal is the standard to be attained; being at or near the 
bottom is something to be avoided. 

The decision to compare a state to the number one ranked state (in this paper) or the top ten 
ranked states (Stoat, reflects two different interests of the policymaker^^ The policymaker 's interest may 
be in how the system is doing, and parttoularly, how far It has to go to reach the top of the pile. Or, it may 
be the case that policy maker wants or needs to explain the relatively k>w standing of a state on a particular 
indicator. For example, the comparison between Arizona and the ten states ranked highest on per pupil 
expenditure done by the Arizona Department of Education explains; 
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By comparing and contrasting these states with Arizona and the national average, it is hoped 
that policymalters, education professionals and the general public will bottpr understand the 
^cdeconomic and fiscal characteristics which determine Arizona's educaiional environment 
ciifd placement in national rankings.^^ 

Arizona's relatively poor perfomance is contextualized by socioeconomic and fiscal characteristics by the 
researcher . For the policymaker, however, the opportunity to explain away poor perfonnance may be 
more important. 

Which States to Compare? The decision concerning which states to compare has implications 
yet is typically given tittle discussion . The federal govemment, for example uses all fifty states , the 
district of Colunrtbia and a national average in its state comparisons charts. As previously mentioned, this 
analysis uses a "neighboring states" perspective, in comparing Arizona to other states in the Southwest 
and the nation. Policy Analysis fopr Califomia Education (PACE, compares Californial with six other states 
on many of its charts and tables, and tells its readers "comparisons are made among large states with 
diversified economies and similar costs of H*.'ing."^° The comparison states include Texas, New Yorit, 
Illinois, Pennsylvainia, and Michigan but little discussion of what makes these comparisons viable is 
included and we take it for granted that they make for good conrparisons. Do comparison states have 
growing populations, declining populations, etc.? Are they sim'.ar in taxing capacity? State comparisons 
which take into account each state's size, geography and economy still overlook unique histoiy and 
circumstances. Have comparison states established different goals for their students and taught 
different curricula, thereby making comparisons of selected outconnes unfair?^^ 

State comparisons do not reveal inconsistencies in the reporting of educational statistics among 
the states. For example, in the reporting of educational expenditures reported in Table 5, some states 
may include categories that are ignored in other states, such as capital outlay. (See Sloat,{i9881, for a 
discussion of this point.) Or states may define high school dropouts differently by deciding to include or 
exclude individuals studying for the high school equivalency diploma (G.E.D.). 
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MtiHIplfl gvtriflnftft Points and Education Trends 

When muhiple evidence in the same direction exists, the ilkelihood of accurate Interpretation is 
increased. For example, Table 5 on per pupil expenditures! Table 6 cn per capita income/school 
expenditures. Table 10 on on percentage of students taking college admissions tests, Table 13 on 
advanced placement candidates and Table 15 on non-promotion, indicate? decline or slower than regional 
or national growth. Taken as a group, the indicators might be interpreted with greater confidence to 
indicate comparative decline and the search for explanation becomes more warranted. 

Demographic indicators may also predict coming ciianges in education programs and practices. 
Table 2, for example, announces coming changes In the percentage of minority students in the Arizona 
public schools. This in turn may have affected the decision to ask questions about and do research on 
primary home language of Arizona pupils (found in Table 2), and the future development of bilingual and 
English as a Second Language programs to sen/e the limited English proficient population. 
Trends in thfl Souma of Funds and Control of Education. A trend that may have been anticipated by 
tooking at education Indicators has to do with educational revenues. Table 4 shows a shift over the fifteen 
years 1970 to 1985 in the source of revenues (federal, state and local) for public elementary and 
secondary schools. As the percentage of revenue receipts at the state level grew, it may have been 
possible to anticipate increased demands for accountability at the state level and the growth of 
educational reform measures at the state level. Whether an education indicator such as the one reported 
above has predictive possibilities or the explanation is merely an after the fact reoonstmction , the goal of 
some degree of predictive usefulness seems wan'ented. 

praduatinn Rate Trend and What to Do About It. Noticing a trend does not yieW automatic insights into 
what to do about the findings. For example. Table 13 on the High School Graduation rate shows that 
Arizona ranks 47th m the nation with a 63% graduation rate in 1986 (down from 64.2% in 1985). The 
immediate implication is that public schools need to work harder to improve graduation rate, l-lowever, 
Green makes the case that in certain circunr^tances, a isMfil. rather than higher graduation rate will allow for 
a more equitable social distribution.'^^ In this view, a tower percentage of graduates may reduce the ability 
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of the enployment sector to penalize non-attalners. In this context, reporting the the percentage of high 
school graduates does not necessarily determine what policy recommendations best address the issue. 

Further, it Is difficult to anticipate is how policymakers will use education rankings to change cr 
reform the system? Will policymakers approach refomi incrementally, addressing attention to only one 
aspect of the system at a time or work to improve the entire education system using multiple 
approaches.^ Arizona seems to have approached educational reform in an incremental fashion, in which 
policymakers have addressed their attention to one aspect of the education system at a time. Efforts at 
reform included teacher testing and teacher residency programs initiated in 1980 with more recent 
initiatives on early childhood, statewide curriculum development and criterion-referenced pupil tests. 
Regardless of the content and scope of reform efforts, greater state level involvement might have been 
predicted as increasing percentage of the funds to support public education shifted from the local to Ihe 
state level. This in turn has affected the types of questions asked and reform measures proposed. 
FACTORS WHICH LIMIT INTERPRETATION OF PUPIL TEST SCORES 

The final section looks at factors specific to the comparison and interpretation of pupil test scores. 
With the expansion of state-to-state comparisons of pupil achievement additional caveats seem in order. 

Caution in the Interpretation of Test Scores 
Murnane argues that the collection of data on student achievement creates a new set of 

questions concerning student achievement.^ Do the tests measure higher- or tower-ievel thinking skills? 

Do multiple choice formats lend themselves to factual recall only? Are the patterns of increase and decline 

of test score averages interpretable? He poses four additional questions: 

Who Takes the Exams? Before nwking comparisons among states, testing programs might 

address other aspects of the testing situation. Who takes the examinations? Response rates of state and 

k>cal schools districts and individual schools and classrooms are often a function of how potentially k)w 

scoring chikJren are excused or exckjded from the tested sample. Are special-needs chikfren or children 

whose first language is not English considered part of the relevant population? 
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What Tvpfls ftf Questions are Asked? What types Of examination questions are used? It would be 
paitlculaily valuable to look at how students solve problems rather than just correct and incorrect 
responses on nxjitiple choice formats. Mumane proposes not restricting student answers to a pre- 
determined set of possibilities in order to examine Ihis aspect of student ability.^ 

What Types of Generalizations wtfiii be Made? Horst, Tallmadge and Wood point to twelve errors of 

interpretation comnfwnly made when looking at student achievement data to make judgments concerning 
school effective ness.^C The issue that stands out is the assumption that pupil achievement scores gains 
or losses can be attributed to what is actually going on in individual classes and schools. They suggest 
that changes in school programs, personnel, facilities, class sizes, and community can all affect student 
perfomriance data. Farr and Fay, in k}Oking at reading trend data in the United States, suggest that 
comparisons of states on test scores is a risky venture because the available data are too mixed up to sort 
and make interpretations concerning the quality of schools and programs.^^ 

Who Determlneg What Is Vaiiri Knowledge? Criticism of examination content limits valid 

interpretation of pupil test scores. The National Academy of Education points out that we tend to measure 
what we can; and over time, we eventually conne to value what we measure.'*^ Since we know how to 
measure reading, writing and mathematics, they become what society values. Teachers may then feel 
pressure to reduce emphasis on topics that are not covered on the tests in order to show good scores on 
the subset of school goals that are included in an assessment. Personal qualities such as a sense of craft, 
a commitment to justice and caring, dedication to the public good, which are difficult to assess, may 
beconrte Ignored. And, textbooks may Increasingly resemble the test-items that are used on these 
examinations. 

The International Reading Association resokJtion on the assessment of reading also raises the 
question of "ownership" over the knowledge in a fieki and how it will be assessed.^^ IRA expresses 
concern that traditional assessment techniques and assumptions ofti;n "drive" instruction in ways that are 
sometimes contradictory to the best available knowledge and practice and that testing programs based on 
inadequate and inaccurate definitions of subject matter prevent accurate interpretation and comparisons. 
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Ellwein, Glass, and Smith (1988) suggest caution when noting that pupil performance data 
lundlon as "symbolic and political gestures, not as Instrumental refonns.'SO They argue for more 
extensive evaluations of the impacts of testing programs on the classrooms and schools affected. 
Comparisons need to be useful for examining the effects of instmctionai variables on school outcomes. 
The depth and detail of data that are needed to analyze the impact of educational processes on student 
performances are laddng in the national rankings; they require smaWer scale research that addresses 
specific instaictional concerns rather than broadly defined educational indicators that are more lil<eiy a 
reflection of a state's wealth than educational success. 

SUMMARY AND CONCLUSION 

The essay began by presenting a description of the education system in Arizona; its form was 
excerpted from national data sets and selected education comparisons and indicators were presented in a 
straightforward fashion similar to many reports received by legislators, nr>embers of state boards and state 
dnpartments of education and other education policymalters. Data were presented on demographics, per 
pupil expenditures, attendance, enrollment, and pupil-teacher ratio. This was followed by discussion of 
educational outcomes including pupil performance, ACT scores, participation in advanced placement 
courses, graduation and high school graduation rates. The source of much of the data was the U.S. 
Department of Education, and in particular, the state-by-state rankings found In the "Secretary's Wall 
Chart." 

After presenting this quantitative portrait of Arizona schools, the attempt was made to show how 
one might mal<e sense of the numbers which portray Arizona's education system. It was argued that 
interpretation of state-by-state ranl<ings allows for nx>dest opportunities for understanding the strengths 
and/or weaknesses of the system. Education rankings are seen as useful as they call attention to certain 
ways of tooking at problems; they assume greater importance as they shape diak>gue, discourse and 
research questtons. The need to insure financial support for public schools is seen as a function of state- 
by-state rankings. Convergence of multiple indicators is seen as warranting further explanation for and 
identification of education trends though such efforts seem risky at best. The essay concludes by 
highlighting some of the difficulties in the interpretation of state rankings of pupil achievement. 
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